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ABSTRACT 

Covert,  cross-border  tunnels  are  a  security  vulnerability  that  enables  people  and  contraband  to  illegally  enter  the 
United  States.  All  of  these  tunnels  to-date  have  been  constructed  for  the  purpose  of  drug  smuggling,  but  they 
may  also  be  used  to  support  terrorist  activity.  Past  robotic  tunnel  exploration  efforts  have  had  limited  success  in 
aiding  law  enforcement  to  explore  and  map  the  suspect  cross-border  tunnels.  These  efforts  have  made  use  of 
adapted  explosive  ordnance  disposal  (EOD)  or  pipe  inspection  robotic  systems  that  are  not  ideally  suited  to  the 
cross-border  tunnel  environment.  The  Counter  Tunnel  project  was  sponsored  by  the  Office  of  Secretary  of  Defense 
(OSD)  Joint  Ground  Robotics  Enterprise  (JGRE)  to  develop  a  prototype  robotic  system  for  counter-tunnel 
operations,  focusing  on  exploration,  mapping,  and  characterization  of  tunnels. 

The  purpose  of  this  system  is  to  provide  a  safe  and  effective  solution  for  three-dimensional  (3D)  localization, 
mapping,  and  characterization  of  a  tunnel  environment.  The  system  is  composed  of  the  robotic  mobility  platform, 
the  mapping  sensor  payload,  and  the  delivery  apparatus.  The  system  is  able  to  deploy  and  retrieve  the  robotic 
mobility  platform  through  a  20-cm-diameter  borehole  into  the  tunnel.  This  requirement  posed  many  challenges 
in  order  to  design  and  package  the  sensor  and  robotic  system  to  fit  through  this  narrow  opening  and  be  able  to 
perform  the  mission. 

This  paper  provides  a  short  description  of  a  few  aspects  of  the  Counter  Tunnel  system  such  as  mobility, 
perception,  and  localization,  which  were  developed  to  meet  the  unique  challenges  required  to  access,  explore,  and 
map  tunnel  environments. 
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1.  BACKGROUND 

Illegal  international  border  crossings  into  the  United  States  through  man-made  tunnels  and  utility  culverts  have 
become  more  prevalent.  In  particular,  the  border  between  Mexico  and  the  southwestern  United  States  is  a  target 
for  drug  smugglers  bringing  illegal  drugs  into  the  United  States.  Many  technological  and  legal  problems  exist 
in  detecting,  securing,  and  decommissioning  these  tunnels.  Often,  these  access  tunnels  are  tens  to  hundreds  of 
meters  long,  can  have  significant  elevation  gradients,  and  may  contain  water  and/or  various  types  of  debris. 

There  are  several  categories  of  tunnels  being  built  to  gain  entry  into  the  U.S.  These  tunnels  range  in  depth 
and  dimensions.  They  can  be  elaborate  (sophisticated,  symmetric,  professionally  engineered),  or  hastily  built 
(hand  dug,  shallow,  unreinforced).  Some  tunnels  are  dug  to  connect  into  city  storm  drain  systems  allowing 
circumvention  of  the  border,  such  as  in  Nogales,  Arizona.  A  few  tunnels  have  been  documented  to  be  over  one 
mile  long.  The  extreme  variances  in  conditions  that  exist  within  the  tunnels,  and  the  fact  that  many  are  used 
sporadically,  make  identifying  their  existence  and  locating  access  points  to  the  tunnels  a  very  difficult  task. 

Previous  work  performed  with  robots  working  in  tunnels  and  mines  for  teleoperation  and  mapping  and  has 
been  leveraged  by  this  project.  Some  of  the  earlier  work,  documented  by  Laird, ^  laid  out  some  of  the  basic  vehicle 
teleoperation  issues  for  tunnel  exploration  and  reconnaissance.  Mapping  tunnels  and  mines  tends  to  be  difficult 
because  of  the  relatively  nondescript  nature  of  the  walls,  which  is  why  many  of  these  tunnel/mine  mapping 
systems  integrate  lidars  into  their  localization  systems.^  Carnegie  Mellon  University  (CMU)  has  performed 
their  mine  mapping  experiments  in  the  abandoned  mines  in  Pennsylvania  using  Simulataneous  Localization  and 
Mapping  (SLAM)  using  a  custom-built  robot  called  the  Groundhog.^  Other  mine  mapping  systems^  have  used 
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description  of  a  few  aspects  of  the  Counter  Tunnel  system  such  as  mobility  perception,  and  localization, 
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stop/start  scanning  with  2D  lidars:  obstacle  avoidance  while  the  vehicle  is  moving,  then  stopping  to  allow  the 
robot  to  generate  a  stable  3D  scan  of  the  environment.  This  3D  data  was  then  used  with  Markov  Random  Fields, 
an  A*  search,  and  C-space  maps  to  generate  3D  maps  and  plan  future  motion.  Niichter  et  al.^  discussed  a  3D 
SLAM  algorithm  designed  to  handle  the  six  degrees  of  freedom  inherent  in  a  subterranean  environment,  reduced 
point  clouds,  and  a  fast  iterative  closest  point  (ICP)  variant  to  locally  and  globally  register  the  point  clouds. 

This  counter-tunnel  effort  was  funded  by  the  Joint  Ground  Robotics  Enterprise  (JGRE)  from  fiscal  year  10-13, 
run  jointly  by  SPAWAR  Systems  Genter  (SSG)  Pacific  and  Air  Eorce  Robotics  Lab  Airbase  Technologies  Division 
(AERL/RXQ)  Tyndall,  and  was  a  complementary  development  effort  to  the  Rapid  Reaction  Tunnel  Detection 
(R2TD)  Joint  Capability  Technology  Demonstration  (JCTD),  which  addressed  the  objective  tunnel  exploration 
requirements. 

1.1  Objectives  and  Scope 

The  project  objective  is  to  develop  and  demonstrate  technologies  that  will  enable  insertion  of  a  robotic  system 
through  a  small  borehole  into  a  suspect  tunnel  cavity  for  the  purpose  of  conducting  precision  mapping  and 
characterization  operations  in  austere  tunnel  environments  (hand-dug  border  tunnels,  caves,  etc.). 

To  accomplish  this  objective,  the  scope  of  the  effort  has  been  divided  into  three  major  technology  thrusts: 

1.  Development  of  an  Unmanned  Ground  Vehicle  (UGV)  mobility  platform  capable  of  insertion  through  a 
maximum  20-cm-diameter  borehole. 

2.  Development  of  a  borehole  deployment  capsule  to  support  insertion  and  retrieval  of  the  UGV  system  and 
provide  a  communications  node  at  the  tunnel  insertion  point. 

3.  Development  of  a  perception  system,  small  enough  to  fit  inside  the  borehole  deployment  capsule  and  provide 
localization,  obstacle  detection,  and  3D  mapping. 

The  scope  of  this  paper  limits  the  discussion  to  mobility  of  the  target  platform,  the  perception  system,  and 
localization.  Topics  such  as  the  borehole  deployment  and  retrieval  capsule,  communications,  additional  tests,  and 
more  detailed  results  have  been  left  out  of  this  paper  and  can  be  found  in  the  technical  report  to  be  published  by 
SSG  Pacific.^ 

1.2  System  Requirements 

•  Deploy  and  retrieve  UGV  through  an  20-cm  borehole  into  tunnels  3  to  30  m  in  depth 

•  Transit  up  to  800  m  round  trip 

•  Traverse  2  horizontal  45-degree  turns 

•  Climb  a  30  cm  vertical  discontinuity  (i.e.,  a  step) 

•  Traverse  a  30  cm  gap  or  crevice 

•  Operate  over  varied  and  rough  terrain  including  mud,  loose  gravel,  areas  of  limited  traction,  and  up  to  20 
cm  standing  water 

•  Autonomously  explore,  localize,  and  map  the  tunnel 

•  Localize  the  entry  within  1  m  of  accuracy 

•  Generate  3D  model  of  the  tunnel  environment  for  characterization,  measurement  and  analysis 


2.  ROBOTIC  PLATFORMS 


2.1  Prototype  Counter  Tunnel  Exploitation  Robot  (CTER)  Platform 

The  Counter  Tunnel  Exploitation  Robot  {CTER)^  built  by  Raytheon  Sarcos  Integrated  Defense  Systems  (IDS),  is 
a  high  degree  of  freedom  (DOF)  UGV  designed  to  address  the  counter-tunnel  mission  and  is  shown  in  figures  1,  2, 
and  3.  Two  of  these  platfoms  were  delivered:  one  to  AFRL  Robotics  at  Tyndall  AFB,  FL  and  one  to  SSC  Pacific 
Unmanned  Systems  Branch,  San  Diego,  CA. 

The  CTER  has  a  camera  mounted  on  arms  at  the  rear  of  the  aft  track,  and  it  can  rotate  180  degrees  for 
360-degree  viewing.  There  are  also  arms  on  the  front  track  for  payload  attachment.  The  robot  was  delivered  with 
a  fiber-optic  tether  for  communication.  This  tether  provides  a  link  for  commands,  status,  video  feedback,  and 
mapping  sensor  data.  Wireless  communication  has  been  explored  for  this  application  using  Cobham  radios.  The 
dimensions  of  the  CTER  is  17  cm  wide  x  172  cm  long  x  17  cm  tall  (no  payload). 

The  CTER  has  an  on-board  real-time  controller  for  joint  control,  drive  motor  control,  and  camera  compression. 
The  unique  joints  and  motors  arrangement  between  the  tracks  of  the  CTER  allow  for  step  climbing,  roll  prevention, 
and  side-by-side  track  operation.  The  robot  is  also  designed  to  be  submerged  in  up  to  20  cm  of  water  (although 
this  was  not  tested  during  this  project). 

There  are  multiple  modes  available  to  the  robot  operator  including:  Find  Zeroes,  Uncommanded,  Zero  Torque, 
Step  Climb,  Anti-Roll,  Bend,  Follow-the-Leader,  Cobra,  and  Tank.  The  Find  Zeroes  mode  is  the  initialization 
procedure  that  the  robot  needs  to  run  every  time  it  is  powered  on.  The  Uncommanded  and  Zero  Torque  modes 
disable  all  motion  by  the  robot,  and  allow  it  to  be  manipulated  manually  with  no  motor  driving.  Step  Climb 
driving  mode  will  successfully  climb  30-cm  steps.  Anti- Roll  mode  is  a  driving  mode  that  offsets  the  tracks  and 
actively  corrects  for  roll.  If  a  condition  occurs  where  the  mode  cannot  prevent  a  roll,  the  robot  will  go  into  the 
Uncommanded  mode.  Bend  mode  is  a  driving  mode  that  allows  the  operator  to  offset  the  tracks  at  varying  angles 
in  order  to  provide  stability.  Angles  of  15,  30,  and  45  degrees  are  available.  When  turning  in  this  mode,  the 
tracks  turn  into  each  other  in  a  C-shape.  Follow-the-Leader  is  another  driving  mode  in  which  the  rear  track 
attempts  to  exactly  follow  the  path  of  the  front  track.  Cobra  mode  raises  the  aft  track  in  order  to  give  the  camera 
a  better  vantage  point.  In  Tank  mode  (figure  3),  the  robot  will  position  the  tracks  in  parallel  with  each  other, 
and  operate  as  a  skid-steer  vehicle. 


Figure  1:  CTER  platform  maneu¬ 
vering  around  obstacles  in  a  nar¬ 
row  tunnel 


2.1.1  CTER  Testing 

The  CTER  was  operated  for  only  a  short  time  beginning  in  February  2013  through  June  2013.  The  multiple 
driving  modes,  run-times,  mobility,  and  terrain  maneuvers  were  all  evaluated. 


Figure  2:  CTER  platform  fitting  Figure  3:  CTER  platform  in  tank 
through  tight  passages  mode 


Testing:  Driving  Modes  and  Run  Times  The  vehicle  run  time  was  evaluated  for  each  driving  mode  in  a 
straight  run,  flat  gravel  surface,  fully  charged  battery,  and  no  payload  installed.  The  vehicle  was  run  until  the 
battery  was  completely  discharged.  The  results  are  shown  in  table  1. 


MODE 

DISTANCE 

TIME 

Anti-Roll 

1356  m 

69  min,  14  sec 

Bend 

1920  m 

115  min,  57  sec 

Follow-the-Leader 

2240  m 

112  min,  42  sec 

Tank 

2156  m 

122  min,  32  sec 

Table  1:  CTER  drive  times  and  distances  in  various  drive  modes 

The  Tank  mode  is  the  most  stable  of  all  the  modes,  however  in  this  mode  the  payload  is  facing  the  rear  of  the 
robot.  The  Anti- Roll  mode  is  currently  tuned  to  prevent  roll  by  the  robot  in  the  basic  configuration  with  no 
payload.  Adding  a  payload  and  running  with  this  mode  causes  continual  overcorrection  by  the  robot.  In  some 
cases  this  causes  the  payload  to  hit  the  ground  on  both  sides  as  the  robot  oscillates.  With  a  payload  installed, 
tests  have  found  that  Bend  mode  and  Follow-the-Leader  mode  provide  the  most  effective  driving. 

The  Step  mode  of  the  robot  was  also  evaluated,  and  it  will  successfully  climb  30-cm  steps.  The  steps  do 
however  need  to  have  a  front  face  for  the  tracks  to  drive  up.  The  robot  will  also  successfully  cross  a  30-cm  gap  in 
all  of  the  driving  modes. 


Testing:  Mobility  The  mobility  testing  of  the  CTER  was  to  evaluate  effectiveness  of  its  ability  to  turn  around 
in  a  tunnel,  climb  up  and  down  30-cm  steps,  drive  in  91-cm  culverts,  turn  around  in  91-cm  culverts,  drive  over 
gaps  of  30  cm,  and  evaluate  effectiveness  of  fiber-optic  spools  for  control  of  the  robot. 

Turning  the  CTER  around  in  a  small  space  was  very  challenging.  It  was  determined  when  driving  in  bend  or 
follow-the-leader  mode  that  putting  the  robot  in  tank  mode,  turning  around,  and  returning  to  the  previous  drive 
mode  was  the  easiest  way  to  turn  around. 

The  CTER  was  able  to  climb  up  and  down  30-cm  steps.  However,  climbing  required  a  flat  floor  surface  and 
sufficient  floor-surface  friction.  Polished  concrete  was  found  to  provide  insufficient  friction  for  the  snake  to  climb 
the  stairs.  Plywood,  gravel,  and  carpet  were  all  adequate  floor  surfaces  to  allow  step  climbing  by  the  CTER. 

Driving  in  91-cm  culverts  was  successful  but  required  a  close  watch  on  the  roll  angle  while  driving.  Since  the 
culvert  had  no  reference  points  to  look  at  while  driving,  the  roll  indicator  was  the  only  way  to  determine  if  the 
CTER  was  driving  at  the  bottom  of  the  culvert.  Without  the  roll  indicator,  roll-overs  of  the  CTER  are  to  be 
expected.  Turning  around  in  the  culvert  was  accomplished  by  changing  to  Tank  mode,  turning  around,  and  then 
resuming  in  the  previous  driving  mode. 

The  CTER  was  able  to  cross  30-cm  gaps  with  no  problem.  Due  to  the  track  length  being  almost  double  the 
30-cm  gap  size,  there  was  no  visually  perceptible  drop  on  the  front  of  the  track  when  crossing  a  30-cm  gap. 

The  CTER  is  a  prototype  robot  design,  built  to  meet  the  requirements  of  delivery  through  a  20-cm  borehole, 
driving  800  m,  and  communicating  via  fiber-optics.  This  platform  was  able  to  meet  these  requirements,  however 
not  without  reliability  issues.  Both  of  the  delivered  CTER  platforms  had  to  have  the  joints  of  the  center  section 
replaced  as  a  result  of  cold  welding  between  the  original  joints.  Additionally,  a  track  controller  circuit  board  had 
to  be  replaced  due  to  a  failure.  The  power  up  calibration  sequence  was  also  sometimes  problematic  as  it  would 
occasionally  initialize  in  the  wrong  orientation.  This  would  then  require  a  reboot  and  rerun  of  the  calibration 
sequence  until  it  was  performed  correctly.  A  couple  months  after  initial  receipt  of  the  AFRL  CTER^  it  had  to  be 
returned  to  the  manufacturer  for  an  upgrade.  This  upgrade  put  a  real-time  controller  on  board  the  CTER  and 
seemed  to  improve  the  responsiveness  of  the  controls.  A  shipping  container  designed  specifically  for  the  CTER 
was  also  built  due  to  problems  encountered  during  shipping  of  the  CTER  at  SSC  Pacific.  The  time  required  for 
these  fixes  and  upgrades  greatly  limited  the  amount  of  time  available  for  testing  and  evaluation  of  the  CTER 
platform. 


Testing:  Terrain  Maneuvers  The  CTER  was  driven  over  varied  terrain  and  performed  well  (without  the 
localization  and  mapping  payload).  Using  the  Anti- Roll  mode,  the  platform  was  able  to  drive  over  gravel,  crushed 
rock,  and  obstacles  such  as  pieces  of  2x4-inch  lumber  successfully  without  tipping  over.  Large  obstacles  such  as  a 
pile  of  15-30-cm  rocks  was  found  to  be  too  difficult  for  the  CTER  to  navigate.  Climbing  over  these  obstacles  often 
resulted  in  roll-over  even  in  the  Anti-Roll  driving  mode.  In  other  modes  such  as  Bend  and  Follow-the-Leader,  the 
snake  was  able  to  successfully  navigate  the  previously  mentioned  obstacles  but  would  require  care  to  prevent 
tip-over.  When  tip-overs  did  occur,  the  CTER^s  Roll  mode  was  used  to  roll  the  robot  to  the  upright  position. 
Corrugated  plastic  and  steel  also  presented  no  traversal  problems. 

2.2  Alternative  platform:  iRobot  PackBot 

SSC  Pacific  created  a  plan  to  utilize  an  alternative  robotic  platform  for  testing  and  evaluation  of  sensors  and 
autonomy  algorithms.  This  was  necessary  to  continue  development  work  while  the  CTER  platform  was  being 
prototyped  and  upgraded.  Although  the  alternative  platform  would  not  fit  through  a  20-cm  borehole,  it  would 
still  be  a  viable  options  for  ISR,  mapping,  and  characterization  of  tunnels  if  other  methods  of  entry  were  available. 

The  iRobot  PackBot  was  the  primary  vehicle  for  sensor  and  autonomy  development.  SSC  Pacific  has  used  the 
PackBot  in  various  other  projects  and  even  developed  a  tele-operation-to-autonomy  conversion  box  that  would  fit 
inside  one  of  the  three  payload  bays,  loaded  with  an  Intel  Dual  Core  processing  board,  KVH  DSP-3000  fiber-optic 
gyro,  MicroStrain  3DM-GX2  IMU,  with  USB  and  ethernet  connectors.  The  PackBot  was  small  enough  to  fit  into 
all  the  tunnels  that  were  explored  and  provided  a  robust  wireless  communication  link  without  any  additional 
radios  attached.  The  dimensions  of  the  PackBot  are  52  cm  wide  x  89  cm  long  x  17  cm  tall  (no  payload). 


Figure  4:  iRobot  PackBot  with  mounted  NREC  MultiSense-SL 


3.  PERCEPTION 

The  Counter  Tunnel  project  had  perception  requirements  to  localize  the  egress  points  within  1  meter  of  accuracy 
after  traversing  400  meters  without  GPS  and  to  create  a  3D  model  of  the  tunnel  environment  for  obstacle  avoidance, 
characterization,  measurement,  and  analysis.  A  3D  model  is  especially  important  for  obstacle  avoidance  because 
the  tunnel  environment  is  so  varied,  consisting  of  stairs,  rocks  and  debris,  sometimes  rail  and  cart  system,  and 
anything  else  it  might  encounter  in  real-world  drug  tunnels.  This  sensor  design  task  was  fulfilled  by  the  National 
Robotics  Engineering  Center  (NREC)  with  their  prototype  3D  sensor  system,  now  called  the  MultiSense-SL. 

3.1  Perception  Sensor 

The  MultiSense-SL^  figure  5,  was  developed  by  the  National  Robotics  Engineering  Center  (NREC)  to  meet  the 
localization  and  mapping  requirements  in  a  small  package.  It  uses  an  axially  spinning  Hokuyo  UTM-30LX-EW 
rotating  about  the  z  forward  axis,  an  optimal  view  for  tunnels,  which  is  calibrated  with  the  left  camera  of  the 
stereo  pair.  Four  strobing  LED  lights  surrounding  the  sensor  provide  enough  lighting  for  accurate  stereo  camera 
analysis  in  no-  or  low-light  conditions  such  as  inside  a  tunnel.  The  software  (running  on  a  hardened  computing 


box  with  an  Intel  Core  il  processor)  computes  the  position  estimate  through  visual  odometry  (VO)  from  the 
stereo  cameras  at  10  Hz.  and  a  position  correction  from  the  iterative  closest  point  (ICP)  algorithm  from  the  lidar 
at  3  Hz. 


Figure  5:  NREC  MultiSense-SL 


Tests  performed  by  SSC  Pacific  demonstrated  that  the  MultiSense-SL  provided  an  accurate  3D  point  cloud 
representation  of  indoor  and  outdoor  environments,  tunnels,  buildings,  and  lab  spaces  (see  figure  6).  However, 
because  visual  odometry  is  the  primary  localization  method  being  used,  position  estimation  becomes  noisy  and 
cannot  recover  when  it  has  no  visual  features  to  detect  or  track  (e.g.  within  a  few  centimeters  against  a  wall). 


4.  LOCALIZATION 

The  vehicles  operating  inside  the  tunnel  are  required  to  do  so  in  a  GPS-denied  environment,  mapping,  localizing, 
and  providing  a  geo-referenced  position  of  the  egress  point  over  400  meters  from  the  entry  with  only  an  error  of 
1  meter.  Many  localization  problems  need  to  be  solved  and  tightly  coupled  to  achieve  this  requirement.  The 
diagram  in  figure  7  illustrates  the  challenges:  GPS  readings  will  be  captured  at  the  entry  point  of  the  borehole, 
the  orientation,  angle,  and  distance  of  the  borehole  apparatus  needs  to  be  calculated  while  moving  through  the 
borehole,  the  vehicle  must  localize  itself  with  respect  to  the  borehole  apparatus  once  it  has  been  deployed,  and 
finally  the  vehicle  will  need  to  accurately  calculate  its  odometry  through  the  tunnel. 

4.1  Borehole  Apparatus  Localization 

One  of  the  challenges  in  creating  an  accurate  3-dimensional  mapping  of  any  tunnel  being  investigated  is  obtaining 
an  initial  geospatial  location  and  orientation.  This  initial  pose  anchors  the  3D  map  in  reference  to  known  locations 
and  entities  on  earth.  Without  an  accurate  initial  pose,  the  projection  of  the  tunnel  and  most  particularly  the 
exit  points  may  have  a  high  degree  of  error.  The  majority  of  the  geospatial  location-providing  devices  use  Global 


Localization  and  Mapping  Accuracy 


GPS  Error 
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+  vehicle  localization  error 
(over  400  m  tunnel  length) 


Initial  Robot  Geo-referenced 
Position  and  Orientation  Error 
when  Deployed  from  Borehole 


Figure  7:  System  localization  error 


Positioning  System  (GPS).  The  tunnel  environment,  which  may  be  as  much  as  30  m  below  the  surface,  prevents 
GPS  units  from  functioning  effectively  due  to  the  weak  signal  received  at  those  depths.  The  approach  was  to  find 
a  positioning  system  that  can  take  a  geodetic  position  while  on  the  surface  of  the  borehole  and  calculate  the 
change  in  position  as  the  delivery  capsule  descends  through  the  borehole.  The  unit  determined  to  be  able  to  meet 
all  the  specifications  was  the  Kearfott  KI-4901  T-24  INU  (Inertial  Navigation  Unit).  Tests  showed  that  after  a 
32-m  vertical  test,  the  3D  position  error  was  only  0.356  m,  azimuth  error  was  0.022  degrees,  pitch  error  was  0.371 
degrees  and  roll  error  was  0.321  degrees. 

4.2  Initial  Robot  Localization  from  a  Fiducial  on  the  Borehole  Apparatus 

Once  the  vehicle  has  been  deployed  from  the  borehole  apparatus,  it  localizes  itself  with  the  apparatus,  which 
should  already  be  localized  with  the  surface  coordinate  system  (normally  GPS).  This  is  done  by  first  detecting 
the  exposed  portion  of  the  deployment  apparatus  using  a  rotationally  invariant  localization  fiducial  attached 
to  the  tip  of  the  apparatus  (figure  8).  References  for  this  technique  can  be  found  in  the  papers  by  Olson, ^ 
Zheng, ^  and  Bradley.^  With  the  known  size  and  pattern  of  the  localization  fiducial,  the  vehicle  can  calculate  its 
six-degree-of-freedom  position  and  orientation  using  only  a  single  image  from  a  single  camera.  This  position  and 
orientation  can  then  be  transformed  through  the  known  apparatus  coordinate  frame  and  localize  the  vehicle  with 
the  surface  coordinates. 


Figure  8:  Deployment  of  the  CTER  platform  through  the  borehole  apparatus  with  attached  localization  fiducial 


4.2.1  Pattern 


The  localization  pattern  is  comprised  of  black  or  white  circles  inscribed  within  oppositely  colored  black  or  white 
squares.  This  allows  the  pattern  to  contain  binary  data  as  well  as  be  easily  localized  using  calibration  techniques. 
The  encoded  data  can,  in  the  cylindrical  case,  encode  the  yaw  rotation  of  the  pattern  as  it  is  viewed  from  different 
vantage  points.  In  the  flat  case,  the  bits  can  encode  the  unique  id  of  the  pattern,  allowing  large  scale  localization 
to  be  done  with  multiple  patterns.  In  either  case,  the  pattern  needs  to  be  rotationally  unique,  e.g.  no  rotation 
of  the  pattern  can  be  equal  to  itself  or  any  other  pattern.  In  the  cylindrical  case  (figure  9),  every  view  of  the 
pattern  needs  to  be  rotational  unique.  If  this  uniqueness  is  not  maintained,  then  localization  ambiguities  will 
arise,  allowing  for  multiple  possible  solutions.  Ensuring  that  the  patterns  are  greatly  different  from  each  other 
allows  for  some  additional  error  checking.  To  ensure  this  uniqueness,  the  targets  use  a  solid  strip  of  black  or 
white  circles  on  the  top  and  an  alternating  pattern  on  the  bottom. 


Figure  9:  5x16  Cylindrical  Localization  Pattern. 


4.2.2  Algorithm 

The  localization  algorithm  is  designed  to  calculate  the  relative  position  of  the  image  with  respect  to  the  pattern. 
The  algorithm  first  detects  possible  bits  using  a  contour  detection  algorithm.  Contours  are  detected  between  two 
threshold  values  and  contours  that  are  found  at  multiple  threshold  levels  and  a  sufficiently  convex  are  added 
as  potential  pattern  bits.  The  values  of  the  threshold  minimum  and  maximum  as  well  as  the  threshold  step 
are  used  to  determine  which  intermediate  threshold  values  will  be  used.  Larger  ranges  and  smaller  steps  will 
make  the  detection  algorithm  more  sensitive  but  take  more  time,  whereas  tighter  ranges  and  larger  steps  will 
be  faster  but  may  miss  some  of  the  pattern  bits.  Once  the  potential  bits  are  detected,  they  are  clustered  using 
hierarchical  clustering  over  both  location  and  contour  diameter.  The  largest  clusters  are  used  to  detect  the 
presence  of  a  bit  pattern  and  organize  the  bits  into  a  grid  structure.  The  grid  is  then  compared  to  the  pattern 
using  different  rotations  and  different  shifts.  Figure  10  shows  the  results  of  these  matching  on  cylindrical  patterns. 
In  the  cylindrical  case,  these  shifts  translate  to  rotations  about  the  cylinder.  The  rotation  and  shift  with  the 
greatest  number  of  fully  matched  columns  is  used  for  localization.  In  the  cylindrical  case,  the  columns  are  the  only 
fully  visible  portions  of  the  pattern.  Ties  are  broken  by  the  total  number  of  bits  matched.  Perspective- n-Point 
camera  pose  estimation  is  run  on  the  pixel  locations  and  the  matched  patterns  3D  model  points,  resulting  in  a 
full  six-degree-of-freedom  transform  from  the  image  to  the  pattern.  Mean  reprojection  error  is  used  to  break 
any  remaining  ties.  When  multiple  unique  patterns  are  being  used,  each  potential  pattern  can  be  matched  and 
ties  can  be  broken  as  previously  mentioned  to  determine  which  pattern  best  matches.  This  transform  can  be 
combined  with  the  known  global  pose  of  the  pattern  to  determine  the  global  pose  of  the  image. 

4.2.3  Testing 

Tests  from  this  method  produced  results  that  were  within  1-3  cm  accuracy  and  less  than  1  degree  azimuth 
angle  error.  These  results  were  determined  using  the  camera  images  from  the  MultiSense-SL  and  the  lidar  scans 
from  the  same  sensor  as  ground  truth.  The  authors  postponed  more  thorough  testing  of  the  fiducial  system 
to  implement  an  April  Tags^  solution  for  comparison.  However,  time  and  budget  restrictions  did  not  allow  for 
additional  localization  implementations  or  more  thorough  testing. 

4.3  Vehicle  Localization 

The  NREC  MultiSense-SL  was  designed  to  provide  perception  and  localization  for  the  system,  but  when  the 
CTER  platform  and  the  MultiSense-SL  sensor  were  delivered,  testing  showed  significantly  degraded  mobility 


Figure  10:  Fiducial  matching  to  tilted  a  cylindrical  pattern. 


performance.  A  second  generation  of  sensor  miniaturization  was  planned  but  didn’t  go  forward  because  of  budget 
cuts.  The  CTER  platform  was  very  unstable  with  the  sensor  mounted  on  the  arms  of  the  front  tracks,  causing 
tip  over.  During  testing,  the  Roll  mode  was  used  to  self-right  the  platform,  but  the  joint  gear  of  bender  3  broke 
a  tooth  with  the  weight  of  the  sensor,  and  the  robot  was  sent  back  for  repairs.  Due  to  this  issue,  testing  of 
localization  and  mapping  were  never  conducted  with  the  CTER  platform. 

However,  localization  tests  were  conducted  with  the  iRobot  PackBot  platform  and  produced  positive  results. 
No  formal  characterization  of  the  accuracy  of  the  localization  of  the  MultiSense-SL  sensor  was  performed  due  to 
budget  and  time  issues.  SSC  Pacific  ran  an  informal  test  of  the  sensor  mounted  on  an  iRobot  PackBot  inside  a 
1.5  m  diameter  concrete  storm  drain,  driving  450  meters  in  and  450  meters  out  (total  of  900  meters  traveled) 
multiple  times  and  the  largest  position  error,  using  only  NREC’s  sensor’s  odometry,  was  0.9  meters.  This  was  an 
impressive  feat,  especially  because  there  aren’t  many  visual  features  in  a  uniform  concrete  storm  drain,  as  shown 
in  figure  11.  A  corresponding  registered  point  cloud  from  the  storm  drain  can  be  seen  in  figure  12. 


Figure  11:  NREC  MultiSense-SL  mounted  on  iRobot 
PackBot  mapping  and  localizing  itself  in  storm  drain 


5.  DEMONSTRATION  AND  TEST  WITH  U.S.  CUSTOMS  AND  BORDER 

PROTECTION 


SSC  Pacific  held  a  test  and  demonstration  event  near  the  border  of  California  and  Mexico  with  the  U.S.  Customs 
and  Border  Protection  in  November  of  2013.  SSC  Pacific  demonstrated  control  of  the  iRobot  PackBot  and  the 


CTER  platform  through  three  concrete  storm  drains.  Storm  drain  7^1  is  a  45-cm  culvert  with  no  angles  and 
length  of  about  10  meters,  storm  drain  #2  is  a  152-cm  culvert  with  one  90-degree  turn  at  20  meters,  and  storm 
drain  #3  is  a  152-cm  culvert  with  one  90-degree  turn  at  30  meters  and  another  90-degree  turn  at  150  meters. 
The  PackBot  was  controlled  through  wireless  communications  displaying  a  video  stream,  and  overlaid  local  map 
data  on  satellite  imagery.  Wireless  communications  were  especially  good  in  the  concrete  structures,  802.11  wifi 
was  used  to  provide  video  to  the  user  and  send  control  commands  to  the  robot  for  up  to  450  meters  away  from 
the  base  station  in  storm  drain  #2.  The  signal  was  lost  after  the  second  turn  in  storm  drain  #3,  about  150 
meters  from  the  base  station.  This  was  expected  from  previous  communications  tests  in  other  tunnel  locations. 
Manually  Deployed  Communication  Relays  (MDCR),  a  variant  of  the  Automatically  Deployed  Communication 
Relays  (ADCR),^^  a  proven  and  fielded  communications  product  developed  at  SSC  Pacific,  provided  repeater 
relay  communication  nodes  that  were  used  to  increase  the  communication  distance.  A  repeater  node  was  placed  at 
the  second  turn  which  provided  strong  communications  to  the  PackBot  for  the  rest  of  the  tunnel  length,  another 
200  meters  (figure  13).  The  autonomy  system  was  successfully  able  to  travel  the  distances  of  the  storm  drains 
with  user  interactions  only  at  the  start,  the  halfway  point  when  the  vehicle  turned  around,  and  the  end. 


Figure  13:  Top-down  view  of  3D  point  cloud  tunnel 
capture  showing  two  turns 

A  test  was  conducted  with  the  CTER  platform  through  storm  drain  #1,  shown  in  figure  15.  The  CTER 
platform  was  connected  with  fiber  and  controlled  using  the  CTER  control  software  with  video  and  position 
feedback  as  shown  in  figure  16.  The  small  diameter  the  storm  drain  provides  a  very  unstable  surface  for  a  single 
track  linear  platform  and  the  CTER  tipped  over  after  traveling  only  3  meters.  The  small  width  of  the  culvert 
made  maneuvering  and  recovering  from  tipover  very  difficult,  especially  only  using  the  video  screen  feedback.  A 
supporting  structure  was  attached  to  the  front  track  section,  as  seen  in  figure  17,  which  allowed  the  platform  to 
successfully  drive  the  remaining  length  of  the  concrete  pipe. 

Additional  tests  were  conducted  at  another  set  of  storm  drains  that  had  varying  degrees  of  ground  cover:  soft 
and  moist  dirt,  loose  sandy  steps,  and  large  rocky  terrain  (figures  18,  19,  and  20).  The  U.S.  Customs  and  Border 
Protection  had  tested  other  platforms  at  these  test  sites  and  stated  that  none  of  them  were  built  to  handle  the 
loose,  sandy  steps.  The  CTER  platform,  with  its  unique  control  and  capabilities,  was  able  to  wriggle  and  rotate 
and  climb  up  the  steps  with  little  effort  in  its  pre-defined  Bend  mode. 

6.  CONCLUSION 

The  Counter  Tunnel  project  has  focused  on  developing  solutions  to  traverse,  characterize,  and  map  a  tunnel-like 
environment.  Work  was  performed  jointly  by  AFRL  {CTER  snake  platform  testing  and  borehole  apparatus, 
deployment,  and  retrieval)  and  SSC  Pacific  (sensors,  characterization,  wireless  communications,  localization,  and 


Figure  15:  CTER  platform  about 
to  enter  a  45-cm  storm  drain 


Figure  16:  The  video  feed  from  Figure  17:  CTER  platform  with 
CTER  platform  as  it  traversed  angled  fins  to  provide  stability  for 
through  the  45-cm  storm  drain  the  45-cm  storm  drain 


Figure  18:  CTER  platform 


traversing  sandy  terrain  in  storm 
drain 


Figure  19:  CTER  platform 

traversing  sandy  stepped  terrain  ^ 

in  storm  drain  Figure  20:  CTER  platform  in 

bend  mode  traversing  rocky  ter¬ 
rain  in  storm  drain 


autonomy)  with  final  test  and  demonstration  events  at  AFRL  Tyndall  Mock  Test  Tunnel  and  San  Diego  border 
region. 

This  project  has  increased  the  knowledge-base  of  viable  counter  tunnel  systems  (platforms,  sensors,  communi¬ 
cations,  and  payloads)  and  has  furthered  the  work  in  GPS-denied  navigation  and  3D  perception  and  mapping. 
The  evaluations  and  expertise  gained  from  this  work  will  be  key  to  providing  robust  and  deployable  systems  to 
address  tunnel,  cave,  and  like  threats. 
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